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ABOUT the CONSORTIUM 
for POLICY RESEARCH 
in EDUCATION 

The Consortium for Policy Research 
in Education (CPRE) unites five of the 
nation’s leading research institutions 
in an exciting venture to improve stu- 
dent learning through research on edu- 
cation policy, governance and finance. 
Members of CPRE are the University 
of Pennsylvania; Harvard University; 
Stanford University; the University of 
Michigan; and the University of 
Wisconsin-Madison. 

CPRE conducts research on issues 
such as; 

0 Education Reform 
0 Student and Teacher Standards 
0 State and Local Policymaking 
0 Education Governance 
0 School Finance 
0 Teacher Compensation 
0 Student Incentives 

Consortium for Policy Research 
in Education 

Graduate School of Education 
University of Pennsylvania 
3440 Market Street, Suite 560 
Philadelphia, PA 19104-3325 
215 573 0700, ext. 0 



ABOUT RESEARCH 
for ACTION 

Research for Action (RFA), a non-profit 
educational research and evaluation 
group located in Philadelphia, works 
with public school districts, educational 
institutions, and community organiza- 
tions to develop programs and policies 
that provide equitable educational 
opportunities and outcomes for all stu- 
dents. RFA conducts research about how 
to ensure equitable educational opportu- 
nities and outcomes for all learners. 
Because we value collaborative consider- 
ation of information, issues, and ideas, 
we also work with educational institu- 
tions and community organizations to 
use inquiry processes to build coalitions 
and to create agendas for action that are 
purposeful and socially just. 

Research for Action 
International House 
3701 Chestnut Street 
Philadelphia, PA 19104 
Phone 215 823 2500 
Fax 215 823 2510 




ABOUT the OMG CENTER 

The mission for the OMG Center for 
Collaborative Learning is to support 
organizational learning and problem- 
solving through consultation, applied 
research and professional education. 
Center staff draw upon recent and estab- 
lished theory within the paradigms of 
systems thinking, action research and 
organization behavior. The Center works 
with individual organizations and inter- 
organizational partnerships, primarily 
in the non-profit and philanthropic 
sectors. Its main focus is on urban and 
social policy issues; the majority of its 
activities are directed at finding innova- 
tive solutions to problems of community 
deterioration, poverty and discrimina- 
tion, environmental quality, and human 
capital. 

OMG Center for Collaborative Learning 

1528 Walnut Street, Suite 805 
Philadelphia, PA 19102 
Phone 215 732 2200 
Fax 215 732 8123 
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ABOUT the ANNENBERG CHALLENGE 



In 1993 philanthropist Walter Annenberg pledged $500 million to help improve the quality of education 
for Americas neediest children. He challenged private donors to match these funds. Nearly 60 percent of 
the resulting Challenge grants have gone to eight of the nation’s largest school districts, including Phila- 
delphia. The Challenge’s goals are to support an unprecedented number of public schools to work directly 
with their local communities; manage their resources in ways that meet the needs of their particular stu- 
dent population; set high expectations for all students; and assess progress through careful and continuous 
review. The Challenge encourages communities to develop their own strategies to reach these goals. Instead 
of giving funds directly to school districts, the Challenge works through nonprofit collaboratives in each of 
its sites, which in turn are supported by staff of the Annenberg Institute. 
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ABOUT the CHILDREN ACHIEVING CHALLENGE 



Many innovative school reform plans have foundered for lack of resources. In February 1995 shortly after the 
School Board adopted Children Achieving , The Annenberg Foundation designated Philadelphia as one of a 
small number of American cities to receive a five-year, $50 million Annenberg Challenge grant to improve 
public education. 

Among the conditions for receiving the grant was a requirement to produce two matching dollars (i.e.,$100 
million over five years) for each one received from the Annenberg Foundation, and to create an independent 
management structure to provide program, fiscal and evaluation oversight of the grant. To assist in meeting 
both these conditions, the District turned to Greater Philadelphia First, an association of chief executives 
from the region s largest companies, to help raise the matching dollars and to provide the oversight required 
by The Annenberg Foundation. A staff was hired, and the Children Achieving Challenge came into being. 

For the Challenge staff, the initial question was how to harness the, at times, fragmented efforts of various 
organizations that work with the School District to improve schools. Such organizations usually focus on 
specific projects but often have been unable to do much to improve the school system as a whole. For 
this reason, Challenge staff have served as catalysts, conveners and coordinators in a massive collaboration 
between internal and external partners. As a result, the Challenge has helped bring the School District 
together with all of its potential partners in a collective focus and a new way of working that can sustain 
itself long after the Challenge is gone. 

Greater Philadelphia First houses the Challenge and provides oversight to it through the GPF Partnership 
for Reform. In addition to its focus on education, GPF provides leadership on other issues important to the 
economic development and quality of life of the community. 

Children Achieving Challenge 
c/o Greater Philadelphia First 
1818 Market Street, Suite 3510 
Philadelphia, PA 19103-3681 
(215) 575-2200 
Fax (215) 575-2222 
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ABOUT the EVALUATION 



The Children Achieving Challenge has charged the Consortium for Policy Research in Education (CPRE) 
and its partners, Research for Action and OMG Center for Collaborative Learning, to conduct a four-year 
evaluation of Philadelphia s Children Achieving initiative. This summary describes findings from Year 
2 (1996-97). The findings emerged from a series of reports on Children Achieving s progress in implement- 
ing four cornerstones of its reform effort: standards-driven curriculum and instruction; a performance-based 
system of accountability; decentralization of decision making; and more effective coordination of student 
support services. The full reports can be obtained from CPREs Publications Office at (215)573-0700, x233. 
They will also be available this summer on the CPRE website at www.upenn.edu/gse/cpre/. 

How the Evaluation Was Conducted 



The evaluation team conducted interviews and observations in 21 schools and 14 clusters; interviewed 
education reform leaders in the District and throughout the city; and administered a District-wide survey 
of teachers. In all, the team interviewed more than 300 people, including 116 teachers, observed nearly 
40 meetings and professional development sessions, analyzed numerous School District documents and 
received survey responses from more than 7,000 teachers and other school staff members. 

What We Were Looking For 

The 1996-97 school year was Children Achieving s second year, but for three-quarters of the District’s 
schools, it was only the first year of the reform. At this early stage, it would not be reasonable to expect 
to find all schools to be making excellent or even steady progress or student achievement being advanced 
everywhere. However, the evaluation team thinks it is reasonable to expect that: 

1. A clear vision of reform is emerging that is shaping decisions at all levels of the District. 

2. Various offices in the central administration are improving the ways they coordinate efforts and 
resources with each other and with key external partners. 

3. New structures, such as the cluster system, the Family Resource Network and the Teaching and 
Learning Network are in place and functioning well. 

4. Schools are beginning to make changes consistent with the reforms, and there are indications that 
reforms are reaching classrooms and affecting students. 

5. The District is monitoring the work of schools and attempting to provide them with the support they 
need to turn the reforms into reality. 

6. There is evidence of increasing support among educators and the public for the improvements. 

7 . The District is demonstrating a willingness to learn from experience, recognizing problems and 
obstacles, and addressing them. 

The findings summarized indicate how well the School District of Philadelphia and the Children Achieving 
Challenge are meeting these expectations. 
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HO Id REFORM LOOKS in PHILADELPHIA 

Standards and Accountability 

A Snapshot of Philadelphia’s Standards and Accountability Systems 

In 1995 Philadelphia’s Standards Writing Teams drafted academic content standards based on those 
developed by national professional organizations, such as the National Council of Teachers of Mathema- 
tics. Teachers and the public had opportunities to review the standards in draft form before the Board 
of Education adopted them. The District was careful to note that the standards should not dictate 
what or how material should be taught. The District also announced (but postponed) plans to develop 
performance standards (the kinds of work students must do to meet the content standards) and opportu- 
nity-to-learn standards (the conditions necessary for achieving the content standards, such as adequate 
resources and good teaching). However, District officials adopted benchmarks on the SAT-9 assessment 
as interim performance standards. The District sought to support implementation of the standards with 
various resources, such as the Office of Best Practices, and activities, including summer professional 
development institutes for teachers and development of model curricula. The cluster system 1 and the 
Teaching and Learning Network, which works through cluster offices, provided support for teachers on 
standards-based instruction. 

The District chose the SAT-9, a criterion-referenced assessment, to determine whether students are meet- 
ing the standards. Since Philadelphia’s standards draw heavily on national standards, the District felt the 
commercially available SAT-9, which was based on national standards, fit well with the local standards. 

The District developed an accountability system for schools that is based on several performance indica- 
tors combined into a Performance Responsibility Index (PRI). The index is calculated using SAT-9 scores 
(which account for 60 percent of a school’s score on the index), promotion and graduation rates, and stu- 
dent and staff attendance. The District set two-year PRI targets for each school, and schools are reward- 
ed, assisted and/or sanctioned depending on their performance. In the District’s plan, schools that exceed 
their performance targets will receive rewards. Reconstitution — in which up to 75 percent of a school’s 
staff can be forced to transfer — is the harshest sanction. 

The Superintendent and his Cabinet also are being held accountable for improving student achievement 
through a set of yearly performance goals in areas such as standards, curriculum and instruction, account- 
ability and assessment, and student support services. SAT-9 scores account for 50 percent of their perfor- 
mance, and their salaries and bonuses are tied to these targets. 
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1 As part of the Children Achieving reform, Philadelphia schools have been organized into 22 groups composed of a comprehensive 
high school and the middle and elementary schools that feed into it. These organizations are called clusters and are made up of 8 
to 15 schools each. Each cluster has staff to provide leadership and support for reform within its feeder pattern schools. 
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Standards and Accountability Timeline 



April-May 1995 


SAT-9 administered in first 6 clusters in Grades 4 and 8 


August 1995 


First 6 clusters formally established 


December 1995 


Standards Writing Teams convened; writing of standards begins 


April-May 1996 


Draft of content standards distributed (English/Language Arts, Mathematics, 
Science and the Arts) for review; 

SAT-9 administered District-wide in grades 2, 4, 6, 8 and 11 


Summer 1996 


First 4 sets of standards reviewed by Standards Review Teams 


July 1996 


Four-day professional development session conducted for teams of teachers on 
standards-based instruction 


Aug.- Sept. 1996 


Second draft of above standards distributed to all teachers for review; Standards 
Curriculum Resource Guides for grades K-4, 5-8 and 9-12 distributed to teachers; 
16 new clusters brought on line 


Oct.-Nov. 1996 


PRI adopted by Board of Education; 

Public Hearings on recommended standards held in all 22 clusters 


December 1996 


First 4 sets of Recommended Content Standards, Benchmarks and Performance 
Examples with minor revisions adopted by Board 


January 1997 


Review copies of draft standards distributed for Health and Physical Education, 
Social Studies and World Languages 


February 1997 


Resource Guide for Standards-based Assessment and Instruction distributed to 
schools; Plans to reconstitute Olney and Audenried High Schools are announced 


April-May 1997 


SAT-9 administered District-wide in grades 2, 3, 4, 7, 8, 9 and 11; 
Second draft of above standards distributed to all teachers for review 


July 1997 


Final three sets of Recommended Content Standards, Benchmarks and Performance 
Examples with minor revisions are adopted by the board; Reconstitution decision 
for Olney and Audenried reversed by arbitrator; Week-long, content-based profes- 
sional development session conducted for teams of teachers (totaling 1,100) on 
content standards in English/Language Arts, Mathematics and Science 


September 1997 


PRI scores made public (232 schools improved, 77 met their targets, and 15 
labeled "low progress”); A second week-long, professional development session on 
content standards is attended by 600 teachers participating in school teams 


October 1997 


Development of Curriculum Frameworks begins; “Low progress” schools are visited 
by school support teams which report findings and recommendations 


January 1998 


Curriculum Frameworks for English/Language Arts, Mathematics, Science and 
Social Studies distributed to all schools; 

SAT-9 scores adjusted to correct error by test publisher; 

Two schools removed from “low progress” list 


April-May 1998 


SAT-9 to be administered District-wide in grades 2, 3, 4, 7, 8, 10 and 11 


August 1998 


First cycle of rewards/sanctions based on PRI to be released 
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Findings about Standards and Accountability 

High awareness of standards. The District has succeeded in making nearly all teachers aware of the con- 
tent standards, and most of Philadelphia’s teachers believe the standards can have a positive impact on stu- 
dent achievement. Likewise, about 80 percent of teachers are in favor of District-wide measures of student 
performance. Since the first set of standards were adopted, the District has distributed supporting materi- 
als to teachers, including the Resource Guide for Standards-based Assessment and Instruction in February 
1997 and Curriculum Frameworks in English/Language Arts, Mathematics, Science and Social Studies in 
January 1998, and nearly 2,000 teachers have attended workshops in standards-based instruction. 

Teachers’ Perceptions of the Standards 

Survey of Philadelphia Teachers, Spring 199 J 



Statement about Content Standards 


Percentage of teachers who 
agreed with each statement 


I understand the purpose. 


83.2 


I believe it has the potential to benefit by students. 


80.8 


I believe that it already has had positive effects in my school. 


57.4 


I believe that it already has had negative effects in my school. 


22.3 


I believe that it has had no effect in my school 


35.7 



The evaluation team believes the District should continue to address teachers’ concerns (see below) about 
the standards and the accountability system by providing additional materials and further opportunities 
for professional development, by better explaining the components of the PRI, and by appointing an 
expert panel to review and monitor the PRI over time. The District’s teaching staff must be convinced 
that the accountability system is fair, and that the District will correct problems in the system when they 
are identified. 

Teachers dissatisfied with initial implementation of standards. The Children Achieving evaluation team 
found that in this second year of implementation, many teachers had serious concerns about the way the 
District introduced the new content standards, and in the early stages, they felt they had received inade- 
quate guidance and preparation from District administration about how to use the content standards in 
their classrooms. They lacked understanding about what standards-driven classrooms look like and needed 
more concrete examples of what was expected of them. 
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The District distributed draft standards in August 1996 and adopted them in December — not enough 
time, said teachers, for broad discussion or realigning the curriculum, especially since teachers lacked 
examples of or materials on standards-based instruction. Within the central office itself, there were 
debates about whether the District should provide prescriptive curriculum resources in the context of 
a reform attempting to decentralize decision-making authority to schools. The uncertainty about how 
much guidance to provide hampered the process of distributing timely information and assistance to 
teachers — as did key vacancies in leadership positions during this critical time. 

Teachers concerned about accountability system. The evaluation team found that most teachers were 
unreceptive to the new accountability system, calling it “unfair” — often because they did not have a clear 
understanding of the PRI. Many teachers felt the administration was holding them accountable for results 
beyond their control. In addition, some of the indicators included in the PRI, such as staff attendance, 
generated controversy because of the way they are calculated. 

Teachers cite misalignment between standards and SAT-9. About one-third of teachers surveyed felt 
the SAT-9 is well aligned with the District’s content standards. The evaluation team believes the survey 
reflects what teachers have heard from Philadelphia Federation of Teachers officials, who have called for 
“clearer” standards, and the absence of any material in the spring of 1997 making clear connections be- 
tween the standards and the SAT-9- The District too has examined the standards’ alignment with the 
SAT-9, and as a result has made some modifications to the assessment and has piloted new test items. 

The evaluation team recommends that local experts continue to work with Harcourt Brace (the publisher 
of the SAT-9) to review and improve the assessment’s alignment with Philadelphia’s standards. 

Improvements in SAT-9 scores. Despite teachers’ concerns over the delivery of the standards and 
accountability system and their desire for more professional development resources, clusterwide scores 
on the SAT-9 and the PRI rose in 1996-97. Two hundred thirty-two schools improved their overall 
index score; 77 of them met their two-year targets one year early. Thirteen schools were designated low 
progress schools because they failed to progress adequately or declined. 

Other issues. Although many schools received higher PRI scores in 1996-97, these improvements were 
questioned by some members of the media, a few school board members and some teachers, in part be- 
cause of misunderstandings of the complex formulas for scoring the SAT-9, for calculating student and 
staff attendance, and for determining student promotion/persistence rates. The designation of two high 
schools for reconstitution sparked controversy, primarily for the manner in which the plan was announced. 
After arbitration, the two schools were not reconstituted. The evaluation team believes that piloting addi- 
tional student performance measures, such as portfolios and course exams, to supplement the SAT-9 may 
help broaden understanding and support for higher standards and performance-based assessment. 
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Teaching Practice 

A Snapshot of Philadelphia’s Vision of Good Teaching Practice 

The Philadelphia standards do not prescribe specific teaching practices, but Children Achieving advocates 
curriculum and instructional approaches that are perceived as supporting student achievement of the stan- 
dards. These include: 

• personalization of the school experience for students that creates positive and productive relationships 
with teaching staff and other students; 

• flexibility in the time it takes for students to work on tasks and master content; 

• individualized instruction that focuses on the learner (as opposed to the teacher); 

• deep (rather than broad) coverage of subject matter; 

• curriculum that reflects the standards, is interconnected and promotes high-level thinking; 

• learning that is active, based on real-world problems, constructivist and open to alternative solutions 
(rather than simply right answers and wrong answers); and 

• collaboration among teachers. 



Findings about Teaching Practice 

Variations in understanding. Through classroom observation, interviews with teachers and the teacher sur- 
vey, the evaluation team concluded that Philadelphia teachers varied widely in their understanding of how 
to implement standards in their classrooms, as might be expected in the early stages of implementation. 

Most teachers believed they understood the standards and felt the standards were clear. But only about one- 
third of teachers believed they needed to change their classroom practice in order for their students to meet 
the standards. The majority of teachers the evaluation team interviewed perceived the standards as “nothing 
new” or simply as subject or topic guides. These teachers used the standards as they would a curriculum, 
checking off items they already “covered” and concluding that they did not need to change their classroom 
practice. In contrast, some teachers were re-thinking the design of curriculum units or placing more empha- 
sis on the mastery of the content and skills defined by the standards. Many others were asking students to 
do more writing and problem solving to prepare them for the SAT-9, which is a more demanding assessment 
than those previously administered in the district. 
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Teacher Opinions About the Content Standards 

Survey of Philadelphia Teachers , Spring 1997 



Opinions 


Percentage Response, by Subject 


English/ 

Language 

Arts 


Math 


Science 


The standards are clear. 


82.9 


82.8 


76.8 


Most of my students will be able to meet the 
standards. 


46.9 


47.1 


44.3 


I will need to modify or revise more than half of my 
classroom instruction to align it with the standards. 


35.8 


37.9 


39.9 


My school has placed substantial emphasis on 
achieving these standards. 


72.2 


73.7 


65.1 


I have received adequate support in implementing 
these standards. 


59.5 


49.1 


52.0 



Teachers’ sense of efficacy. Part of the reluctance to change teaching practice stems from a widespread 
belief among teachers that conditions of students’ lives outside the classrooms are primarily responsible 
for their poor achievement, and that instructional practice alone is not enough to overcome such problems. 
(Urban teachers around the country often express such views.) Skepticism about the ability of Philadelphia 
children to reach the standards due to these obstacles caused many teachers either to dismiss the new stan- 
dards, assessment and accountability systems as unrealistic, express frustration about what to do and ask for 
more guidance, or call them unfair. On the survey, 71 percent of teachers reported that they believe that 
their success or failure in teaching is due to factors outside of their control. 

Requests for more guidance. Many teachers struggled to understand how standards apply to their daily 
work in the classroom. They were frustrated by the lack of time for discussing the standards and their 
implications on curriculum. They also reported inadequate support and resources from administration to 
do so. Teachers told the evaluation team that they want more specific illustrations, more guidance and 
demonstration lessons so they could gain a clearer sense of how to put standards into practice. The evalua- 
tion team recommends that the School District increase support for teachers in developing curriculum and 
in helping them to screen and obtain appropriate materials. In addition, the District should give teachers 
more opportunities to both interact with teachers who are experienced in standards-based instruction and 
to examine student work and standards-based curriculum units. 
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Stanford Achievement Test and Performance Responsibility Index 
1997 Scores and Changes Since 1996 
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SAT-9 Reading 
Percent at or 

m 

above Basic 


SAT-9 

Mathematics ^ 
Percent at or 
above Basic 


SAT-9 Science 
Percent at or 
above Basic 


Total 

Performance 

Responsibility 

Index 2 


Administrative 

Cluster 


1997 


Change 

Since 

1996 


1997 


Change 

Since 

1996 


1997 


Change 

Since 

1996 


1997 


Change 

Since 

1996 


Audenried 


37.3 


6.8 


23.6 


7.9 


28.4 


7.6 


63 


6.6 


Bartram 


44 


10.3 


24.8 


7.2 


29.3 


9.2 


66.2 


8.5 


CHAIN 


63.2 


10 


47.1 


7.4 


47.9 


9.6 


73.6 


7.4 


Edison 


36.7 


7.8 


23.1 


6.3 


24.5 


7.7 


59.6 


7.5 


Fels 


54.2 


5 


35.6 


3.8 


35.3 


4.7 


68.2 


5.7 


Frankford 


54.6 


8.6 


34.8 


6 


37.7 


6.7 


68.8 


5.5 


Franklin 


49.4 


12.8 


33.1 


8.1 


32.9 


8.2 


66.5 


8.5 


Furness 


53.8 


4.8 


39.7 


9.2 


37.7 


9.5 


69.1 


5.9 


Germantown 


56.5 


4.9 


32.2 


3 


33.7 


9.7 


69 


5.2 


Gratz 


33.3 


8.8 


20 


7.7 


20.6 


5.5 


58.5 


6.4 


Kensington 


49.1 


6 


32.8 


1.9 


35.5 


3.7 


65 


6.2 


Lincoln 


57.3 


5.8 


34.7 


3.8 


38.3 


3.9 


68.2 


5.7 


ML King 


50.7 


2.1 


34.8 


5.9 


33.1 


6.3 


66.6 


3.5 


Northeast 


66 


10.4 


44.3 


5.5 


45.6 


7.1 


74.3 


7.9 


Olney 


42.3 


8.1 


26.1 


8.6 


27.7 


8.4 


61.7 


6.5 


Overbrook 


45.9 


11.5 


22.5 


3.6 


27.1 


5 


63.7 


6.8 


Roxborough 


56.5 


12.7 


29.9 


6.2 


31.9 


5.2 


67.8 


8 


South Philadelphia 


50 


3.7 


32.3 


3.7 


34 


5.2 


65.8 


3.9 


Strawberry Mansion 


34.6 


2.6 


17.7 


0.3 


23.9 


5.6 


59.2 


1.2 


University City 


40.7 


10.7 


17.6 


4.3 


20 


4 


60.1 


6.4 


West Philadelphia 


39.2 


5.2 


22.6 


5.4 


25.7 


4.3 


62.1 


4.7 


William Penn 


41.4 


9.6 


22 


9.5 


17.2 


6.3 


60 


9.1 


District Elementary 


53 


7.1 


35.3 


6.7 


39.6 


7.8 


70.3 


5.7 


District Secondary 


34.9 


9 


14.8 


2.8 


8.7 


3.5 


48.3 


7.5 


Overall District 


48.1 


7.5 


29.8 


5.6 


31.4 


6.6 


65.7 


6.1 



I 1 The shaded clusters are in the first cohort. 
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Most instructional approaches are “transitional.” Through classroom observation and the teacher sur- 
vey, the evaluation team found that most'Philadelphia teachers’ instructional practices can be described as 
“transitional.” That is, at this point many teachers are beginning to change their teaching practice to sup- 
port standards by supplementing traditional methods — such as teacher-directed activities and discussion, 
question-and-answer sessions and drill-and-practice activities — with “constructivist” or standards-based 
methods, such as small group activities, open-ended discussions and responsive writing or journal entries. 

Efforts to improve SAT-9 scores underscored this finding. The evaluation team learned that many teachers 
devoted classroom time, particularly in early spring, to preparing students for the assessment. Teachers 
recognized that the SAT-9 demands that students develop critical-thinking and problem-solving skills, 
and many attempted to take this into account in their classroom instruction. However, their practice 
did not always reflect their intentions. Sometimes they used the old drill-and-practice techniques, which 
are not effective in preparing students for the kinds of open-ended tasks on the test. Nevertheless, 
Philadelphia schools showed improved SAT-9 scores in 1997 compared to 1996. (See chart on page 11.) 
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Decentralization 



A Snapshot of Philadelphia’s New System 

School level. Local school councils of teachers, parents and administrators are being created to govern 
each school’s policies and resources. Small learning communities, or “schools-within-schools,” consisting 
of 400 or fewer students are being developed to provide more personal, responsive and caring instructional 
environments. They are empowered with (and accountable for) decision-making responsibility and resource 
allocation. 

Cluster level. Philadelphia has created 22 clusters of schools, each consisting of a comprehensive high 
school and its feeder middle and elementary schools. Cluster offices are expected to serve as the locus of 
professional development, instructional leadership (through the Teaching and Learning Network) and 
social services. 

Central administration level. The central office is responsible for setting up District-wide standards 
and accountability systems; monitoring schools to ensure equity; providing examples of best practices; 
and solving problems using a customer-focused, service organization model. 

Findings about Decentralization 

Schools: teachers more involved in decision making. Because more decision making is pushed to the 
school level under Children Achieving s decentralization, the evaluation team examined how teachers felt 
about decision making in their schools. As the following table demonstrates, many teachers reported a 
high level of collaboration and some shared decision making with administrators at their schools, but most 
felt they have gained more influence over classroom-related decisions than schoolwide decisions. A majori- 
ty of teachers saw their principals as clear communicators, but fewer — especially middle school teachers — 
viewed principals as effective managers or instructional and organizational leaders. 

Local school councils: function unclear. At this early stage, implementation of local school councils 
was proceeding unevenly. Most schools have multiple and sometimes overlapping or competing decision- 
making structures. Principals need to play a key role in linking these structures and helping to focus their 
work, but they do not always have the vision and facilitation skills to do so. There are few incentives for 
establishing councils because they lack the legitimacy and authority necessary for effective school-based 
management, in part, because the current agreement between the District and the Philadelphia Federation 
of Teachers limits their role. 

Small learning communities (SLCs): signs of improvement. SLCs are spreading rapidly across the Dis- 
trict. Teachers surveyed had high hopes that SLCs can help to improve teaching and learning, and the eval- 
uation team observed that the SLCs that designate leaders and provide them with release time offer better 
support for teachers and students. At this stage of implementation, SLCs take a variety of forms and do not 
always conform to the District’s model in terms of developing instructional themes that cut across subject 
matter or of partnering with outside institutions. 
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Teacher Involvement in Decision Making 

Survey of Philadelphia Teachers, Spring 1997 



Statement 


Percentage of teachers who agreed 
with each statement, by school level 


Total 

Percentage 


Elementary 

Schools* 


Middle 

Schools 


High 

Schools 


The principal, teachers and staff collaborate 
to make this school run effectively. 


73.1 


60.6 


6 2.7 


68.1 


Teachers are involved in making the 
important decisions in this school. 


56.4 


50.6 


37.8 


50.4 


Teachers have a lot of informal opportunities 
to influence what happens here. 


56.3 


50.8 


45.2 


52.3 


Most teachers are active in decision making 
and/or planning committees. 


29.5 


20.9 


15.8 


24.3 



* Includes K-8 schools 



Clusters: defining roles. Philadelphia s 22 clusters vary widely in terms of funding available to cluster 
offices for staffing and school supports (ranging from $81 to $475 per student). Cluster leaders have 
taken different approaches to guiding school improvement; some believe the cluster should offer strong 
instructional direction, while others believe schools should decide what constitutes good practice with 
the cluster office responding to their requests for support. For the most part, clusters are not developing 
coherent strategies to support school improvements that are based on a holistic assessment of where 
schools are. Depending on funding and staff background, cluster offices tend to take on one of four pri- 
mary roles: disseminating District policies and priorities; providing customized, school-based technical 
assistance; developing networks for teachers, administrators, support personnel, parents and students; or 
serving as brokers to connect schools with outside agencies, coalitions and other resources. The evalua- 
tion team found that cluster staff show a high level of commitment to Children Achieving, but that the 
effectiveness of cluster staff could benefit from increased use of professional development strategies for 
strengthening schools’ decision-making processes and use of data. 

Central administration: some progress but work remains. The Cabinet has been expanded to include 
the Executive Committee (made up of high-level district staff and the executive directors of the Phila- 
delphia Education Fund and the Children Achieving Challenge) plus all 22 cluster leaders and key 
central administrators. During the 1996-97 school year, most Cabinet members reported an increased 
seriousness of purpose in their work and improved coordination among offices and programs, although 
cluster leaders were often dismayed that decision making was still operating in a “top-down” mode. 

The Children Achieving Challenge has remained an integral part of the District’s reform efforts. Its seven 
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Work Teams, each of which is made up of staffers from central office, clusters, the Philadelphia Federa- 
tion of Teachers and sometimes schools and other partners, made strong progress in coordinating profes- 
sional development and budgets during the 1996-97 school year. Likewise, the Philadelphia Education 
Fund played a key role in assisting with standards development and channeling resources to clusters. 

The central office struggled to understand its role in a decentralized system during 1996-97. For exam- 
ple, staffers could not reach consensus about the central offices role in guiding the implementation of 
standards or assisting schools with the process of decentralization. Shifting to a customer-focused stance 
also proved difficult. The central office created a School Support Network that is supposed to be a direct 
link between it and clusters and schools, but it initially encountered problems in defining exactly how 
the network would work. Finally, the District made little progress in two important areas: developing 
a system for effective recruitment, selection, professional development and assessment of principals and 
developing and distributing best-practices resources to help schools implement standards. 

The evaluation team recommends that District leaders re-visit the decentralization elements of the 
Children Achieving theory. They should articulate a clear vision for governance that takes into account 
current realities (such as existing agreements with the teachers’ union) and lays out steps for moving 
forward. The School District should develop standards and benchmarks for best practices in developing 
a “customer-focused” central administration, professional development, coordinating budgets, guiding 
school improvement, and using data. Recruitment, selection and professional development of school-level 
leadership should become a top priority. 
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Student Support 

A Snapshot of Student Support 



The Family Resource Network (FRN) is the District’s primary mechanism for revamping student support 
services. The main goal is to diminish the District’s role as a service provider and boost its role as coordi- 
nator, linking a cluster s in-school resources, such as nurses and guidance counselors, with community ser- 



vices, such as social workers and youth centers. At the central administration level, an FRN management 
team works with about 20 specialists. Each cluster is supposed to have an FRN coordinator, but due to 
budget constraints, only nine did in 1996-97. Schools continue to have traditional student support staff, 
including guidance counselors and nurses, who work with cluster FRN staff. 

Findings about Student Support 

Challenges of restructuring. In these early stages, the District faced some difficulties in implementing its 
ambitious structural changes aimed at decentralizing authority for student support services and diverting 
funding from central administration to clusters and schools. These changes were intended to give schools 
more decision-making power and to reorganize the work of support professionals around four clear goals — 
improve student attendance, student health, school safety and family and community involvement — each 
with targets. 

Fiowever, not all of the budget savings at the central office level filtered down to the school level as origi- 
nally intended, but were used instead to offset the District’s budget deficit. One result of this was that 
cluster-level student support staff tended to view decentralization negatively, accusing the central office of 
“dumping” additional responsibilities on them, particularly in the area of special education, without giv- 
ing them additional resources to do the job. 

School-based professionals also felt that their responsibilities increased due to a reduction in the number 
of supervisors. The District’s success in educating them about new roles was limited in 1996-97. Student 
support staff who worked in clusters with FRN coordinators experienced less anxiety about the changes 
and fewer feelings of isolation, but they were no more deeply engaged in the Children Achieving vision 
than their counterparts in clusters without FRN coordinators. 

In addition, the District’s focus on separating support services from instructional components (with the 
well-intentioned aim of freeing teachers from duties that distract them from teaching) created problems 
by denying the inherent links between these two areas. The evaluation team recommends that the FRN 
should place more emphasis on the role of classroom teachers in accommodating students with complex 
social and educational needs. 
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Defining the role of the FRN. FRN central office staff made significant progress in building stronger 
relationships with other city agencies during the 1996-97 school year, and in interviews with the evalua- 
tion team, both school-level student support staff and leaders of other city agencies expressed positive 
views of the goals of the FRN. However, in these early stages of the initiative, they also struggled to 
translate the goals of the Family Resource Network into practice. Leaders of other city agencies wondered 
what it would mean for the District to reduce its involvement in social service provision; school level 
staff often did not see the need to alter their practice. Under the new FRN structure school-level student 
support staff had better access to professional development opportunities than they had in the past, but 
these efforts were still insufficient in terms of time and resources to help them think of their jobs differ- 
ently. The evaluation team believes the District should seek to help school-level staff redefine their roles 
through improved communication and more intense professional development opportunities. It also 
should continue to make efforts to better define its role in relation to other agencies that serve children 
and families in the city. 

Progress in health care and attendance. The School District did make progress in most of the FRN goal 
areas during 1996-97. In particular, the FRN helped more than 2,500 students obtain health insurance 
and primary care physicians. Average daily attendance rose to 86 percent of enrollment, and more than 
4,000 new volunteers became involved in the schools. 
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CONCLUSION 



In the section About the Evaluation (page 4), we identified seven expectations that the public might have 
about the work of the School District in the context of the Children Achieving reform at the end of its 
second year. Overall, the School District of Philadelphia moved forward in each of these seven areas during 
1996-97 (see below), but the evaluation team has defined four constraints that have influenced or impeded 
progress: 

• The District’s central office has struggled to find the right balance between providing sufficient 
guidance to schools while trying to decentralize decision making down to the school level. 

• Communicating effectively to teaching staff and the public about the complex elements of the 
Children Achieving reforms has proved to be a challenge. 

• Expectations for teachers and other staff members have changed significantly, requiring fundamental 
shifts in their practice and their conception of their roles which takes training, on-site support and time. 

• The School District had shortages of experienced personnel (especially middle managers at the school 
level, such as principals and assistant principals) to guide and support the Children Achieving reforms. 

In light of these constraints, we address each of the seven expectations laid out earlier in this report below. 

1. Is a clear vision of reform emerging that is shaping decisions at all levels of the system? 

The central office and the Districts partners worked hard during the 1996-97 school year to develop goals, 
plans and supporting materials to articulate the vision of Children Achieving . Clusters are important sites 
for communicating this vision, although they vary in their approaches. The evaluation team found that 
understanding of the Children Achieving vision was uneven, and this affected the way decisions were 
shaped at all levels of the system. Staff at various levels do not fully understand the implications for prac- 
tice or how to define their new roles. As a result, many do not understand how profoundly their practice 
must change. 

2. Are various offices across the District improving the ways they coordinate efforts and resources 
with each other and with external partners? 

The Children Achieving Challenge’s Work Teams helped better coordinate professional development 
opportunities with outside partners (particularly the Philadelphia Education Fund) in 1996-97 than dur- 
ing the previous year. The District also made progress in coordination within the central office, although 
the finance and personnel departments resisted the “customer-focused” stance to which others in the cen- 
tral office aspired. The evaluation team also found that multiple initiatives in the District often made 
competing demands on cluster staff and schools. 
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3. Were new structures, such as the cluster system, the Family Resource Network and the Teaching 
and Learning Network in place and functioning well? 

The cluster structure was expanded from six clusters in the first year to 22 clusters (the entire District) 
during 1996-97. With all the clusters up and running, the Teaching and Learning Network was fully 
implemented across the District, although it was not fully staffed until January 1997. The Family Re- 
source Network also expanded its reach, although not every cluster was able to hire a coordinator during 
the 1996-97 school year. 

The evaluation team found that these structures were the base for much of the energy and enthusiasm 
for Children Achieving , although their roles and purposes were still being refined and school staff were 
still learning how to use their services. Clusters varied widely in their approaches to their guidance 
role; the FRN had developed concrete objectives, but its overall approach was poorly understood among 
school-level staff; and the TLN was trying to overcome a late start in the enormous challenge of helping 
teachers shift to standards-based instruction. 

4. Were schools beginning to make changes consistent with the reforms, and were there indications 
that reforms are reaching the classroom and affecting students? 

Schools were beginning to implement the reforms in the 1996-97 school year. For example, almost all 
schools were experimenting with small learning communities, attempting to meet the requirements of 
the District as they perceived them. Nearly every school made efforts to prepare students for the SAT-9 
test and encouraged higher student and teacher attendance to boost scores on the PRI, although their 
strategies varied significantly. Teachers were aware of the standards, although classroom practice was 
changing only incrementally as of Spring 1997. Schools put a lot of effort into finding curriculum mate- 
rials and textbooks that aligned with the standards. Schools were implementing the standards unevenly, 
and their efforts were often seriously undermined by constraints such as weak leadership. 

5. Is the District monitoring the work of schools and attempting to provide them with the support 
that they need to turn the reforms into reality? 

Children Achieving s primary monitoring mechanism, the Performance Responsibility Index (PRI), was 
put in place in the 1996-97 school year. Schools participated in the second round of SAT-9 testing in 
Spring 1997, and the District implemented a school review process for low progress schools in Fall 1997, 
which we will discuss in next year’s report. Our research from 1996-97 showed that the PRI is not wide- 
ly understood, and there is significant disagreement about its assumptions as a measurement tool, factors 
which would seem to limit its effectiveness. 
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6. Is there evidence of increasing support among educators and the public for the improvements? 

We have no data on the public’s view of the reform initiatives. Many of Philadelphia’s business and civic 
leaders seem to be encouraged by the seriousness of the School District’s reform effort and the opening 
up of the system to partnerships and collaboration. The survey data collected in 1997 show that teaching 
staff support the overall goals of reform and many, but not all, of its components. During the 1996-97 
school year, significant disagreement existed about key pieces of the reform between the District and 
important partners, particularly the Philadelphia Federation of Teachers, although this relationship had 
begun to improve by summer. Here, too, we see the complexity of the reform and the challenge of com- 
municating concretely about it as hindering progress in obtaining support. 

7. Is the District demonstrating a willingness to learn from experience, address problems, and 
recognize and correct mistakes? 

The District showed a willingness to seek out feedback and address problems. For example, the central 
office coordinated and designed professional development activities with key partners, more effectively 
integrated central office and field efforts, and responded to teachers’ calls for support on the standards 
in the form of the Curriculum Frameworks, which it distributed in January 1998. Teachers’ progress in 
implementing standards-based instruction will depend largely on the District’s ability to diagnose and 
respond to past and current issues and its recognition of the need for significant and strategic support 
through professional development, school-level leadership and supportive school conditions. For progress 
to continue at a reasonable pace, the District also must address the four constraints identified above. 

Finally, the District has been fully cooperative with the evaluation team, providing access to key person- 
nel and information. The District has also shown an openness to feedback from the evaluation team and 
a willingness to discuss and address implementation problems identified by the evaluation. The teacher 
survey data also has been widely used within the district to assess conditions and set priorities. Overall, 
the behavior of District and school personnel indicates a strong desire to learn from experience and 
improve their performance. 
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